When a network, industrial control system, Internet of Things (IoT) deployment, application support infrastructure, or other complex system is being tested for troubleshooting, routine exercise, commissioning, performance verification, security certification, etc., it is often necessary to provide test input. This input could be stimulus in the form of test cases that yield expected results, test data sets to validate performance under load, security challenges, exercises for various corner cases, or abnormal inputs. During testing, the system or network, as well as various systems associated with the system or network, act upon the stimulus and generate a multitude of data outputs (i.e., various data sets, sensor measurements, and observations).
Often, during testing, an analytics system monitors the system's outputs in attempt to determine if the system is performing within prescribed limits. That is, an analytical system may apply various analytical algorithms to the data outputs to correlate and/or organize the data outputs as needed. Recently, machine learning techniques have been applied to these analytical algorithms to allow the algorithms to be continuously tuned over time and to optimize the performance and accuracy of the analytical algorithms. However, analytical techniques, even when applied in combination with machine learning techniques, are purely responsive to network outputs and do not initiate or cause these outputs. Instead, the network or system inputs (i.e., test stimuli) cause a system or network to react or respond in different manners and to produce particular data outputs necessary to complete the test processes.
Consequently, if a network administrator wants to complete different tasks or test a specific aspect or portion of a network (i.e., to test various corner cases, error log entries, loading scenarios, etc.), the administrator needs to generate new inputs (i.e., new test stimuli) that are suitable for the new task and/or for stimulating the specific aspect or portion of the system or network. Unfortunately, generating test stimuli is a laborious task and, thus, even if the analytical techniques are relatively efficient and automated in and of themselves, it may be inefficient to utilize these analytical techniques at least because generating the stimuli to, in essence, activate these analytical techniques is an inefficient process, in terms of both time and cost. In fact, often, the system or network under test has evolved (i.e., changed in terms of users, applications, nodes, etc. to the point that test stimuli are obsolete) before new stimuli can be generated for a new test. That is, user-generated stimuli may react poorly to rapidly evolving systems and network.